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ABSTRACT 



This study explored the use of an inquiry-based interactive 
multimedia program to identify the process by which students' higher order 
thinking skills developed in a science classroom. Videotapes of students' 
interactions with computers were analyzed to identify the use of higher order 
thinking skills. The Macintosh multimedia research package "VideoSearch" was 
used to analyze digital video. The study concludes that when students 
presented their investigations to the whole class, the role of higher order 
thinking skills became evident. Through reflecting on their experiences, 
students were better able to make inferences about the information provided, 
interpret new information about the program and its content, analyze 
available information, make judgments about its relevance, and synthesize 
what they had learned to generate new knowledge. (Contains 17 references.) 
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Video Analysis: Adding Another Dimension to Qualitative Research? 



The study explored the use of an inquiry-based interactive multimedia program, to identify 
the process by which students' higher order thinking skills developed in a science 
classroom. Videotaping students' interactions provides a means to identify the use of higher 
order thinking skills. In order to analyze students' interactions and identify the use of 
higher-order thinking skills appropriate tools and strategies were required. A new software 
package VideoSearch (Knibb, 1997; Maor & Knibb, 1999), a Macintosh multimedia research 
tool for analysing digital video, was used to analyze classroom observations. In this study 
coding categories based on inquiry learning were developed prior to the video analysis. 
Additional categories were developed while the researcher viewed the video segments and 
were based on student interactions with the constructivist-oriented multimedia program. 

Rationale 

An adaptation of Ennis' (1993) "critical thinking assessment" was used in this study as a 
framework to analyze the development of higher order thinking skills. Higher order 
thinking involves inference, interpretation, analysis, judgement, active mental construction 
and synthesis (Resnick & Resnick, 1992; Newman, 1990). Research suggests that higher 
order thinking may be enhanced when students are actively engaged in critical reflection 
within a meaningful context (e.g., Blanton, Moorman & Trathen, 1998). In a constructivist- 
oriented, inquiry-based learning environment small groups of participants engage in critical 
reflection and collaboratively determine a problem, generate creative questions and design 
investigations accordingly (Maor & Taylor, 1995) in meaningful contexts.An inquiry-based 
learning environment may be well supported by the use of interactive multimedia which 
promotes small group work involving specific interactions and scientific investigations 
(Maor, 1999). 

Social negotiation between pairs of students provided ongoing and immediate feedback on 
the students' higher order thinking skills. Open-ended challenges were provided during 
interactions with the program and curriculum material which included a guide to the 
program and questions based on its content. A presentation by pairs of students to the 
whole class informed the teachers and other students of the extent to which participants 
were successful in using their higher order thinking skills. This activity also allowed 
teachers to identify the appropriate student outcomes within the science curriculum 
learning area (Maor, 1998). The use of the inquiry-based interactive multimedia program 
also provided information for further research into student learning and development of 
higher order thinking. Videotaping student interactions in an inquiry-based interactive 
multimedia learning environment provides a means by which to identify the use of higher 
order thinking skills. 

Methodology 

Erickson (1998) provides a methodological link between an inquiry-based classroom 
learning environment and the process of research to identify the students' use of higher 
order thinking skills. The methodology suggested by Erickson (1998, pll55) enabled the 
researcher "to identify the nuances of subjective understanding that motivate various 
participants in a setting [such as a school classroom and] to identify and understand change 
over time" across two collaborative investigations. Both the subjective understanding of 
student motivations and changes in thinking skills over time have been found to be 
significant in the development of higher order thinking skills (Ennis, 1993; Lewis & Smith, 
1993) 

Twice each week during the month following the initial investigation, students interacted 
with each other and the multimedia program. Video taping of classroom interactions 



provided a means of analysing students' thinking skills within the rich context of the whole 
classroom learning environment. Analyses of the video data were facilitated by the use of 
the VideoSearch program (Knibb, 1997, Maor & Knibb, 1999). Trustworthiness of the data 
was enhanced by the use of VideoSearch program as it enabled the long-term storage and 
retrieval of indexed data for analyses and an efficient and effective audit trail. This offers 
the potential to provide a ready means of reaccessing the data by the researcher for 
presentation and critical review by colleagues to ensure credibility of the data, the analyses 
and interpretations. Information about the process of students' learning while generating 
collaborative scientific investigations can thus be transferred and applied to other students 
in the same classroom and across different classrooms and schools. 

Analysis of the data 

In this paper, a case study of one pair of students, Louise and Elise, is presented. The 
students' learning is analysed during their interactions with each other while using an 
inquiry-based interactive multimedia program. Birds of Antarctica (Maor & Phillips, 1996), 
during a presentation to the whole class and in an interview about their experiences. 
Episodes were created from segments of video which comprised self-contained interactions 
between students and/or the multimedia program and/or the researcher. Together, these 
episodes provide a broad and a contextualised perspective on the development of higher 
order thinking skills. 

Overview of the VideoSearch Data Display for This Study 

An investigation undertaken by the students, a student presentation and a student interview 
each constituted one episode. The digital movie shown in the top right hand corner of 
Figure 1 is a combination of two movies. The left-hand side of this movie shows what part 
of Birds of Antarctica (Maor, 1996) the students were viewing at a particular time. The right 
hand side of the movie shows a synchronised view of the students interacting with each 
other and the multimedia program. These movies were digitised separately, then joined 
together to form a single digital movie with the two views of events occurring at the same 
point in time. 
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Figure 1: An example of a general VideoSearch display of the collated data 



Coding episodes included establishing observed categories, deciding the length for each 
instance and viewing, transcribing and writing the description for each instance (see Figure 
1). This required around four hours for fourteen minutes of observation time. The coded 
categories (e.g., generate a question, hypothesis, etc.) for a single episode appear in a 
separate code window under the relevant QuickTime movie. The categories presented in 
Figure 1 constitute the components of a scientific investigation. While generating each of 
these categories, the researcher identified chronological instances of those observed 
categories. The initial text of these instances appears to the right of the categories in Figure 1 
with the duration of the instance identified by the length of the box. Each of these 
chronological instances consists of the description of that instance (e.g., dialogue between 
the students and some researcher comments). 



In this study, coding categories based on inquiry learning were developed prior to the video 
analysis. Additional categories were developed while the researcher viewed the video 
segments and were based on student interactions with a constructivist-oriented multimedia 
program. From the general display of the collated data, the researcher was able to focus on 
and profile specific investigations by two students. 

Analysing Higher-Order and Critical Thinking Skills of Two Students 

A profile is a visual representation of the process of a scientific investigation. Initially, 
profiles of student investigations were relatively simple, for example. Investigation I by 
Louise and Elise as displayed in Figure 2. 
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Figure 2: Complete VideoSearch display of collated data for Investigation I by Lousie 

and Elise 



In this episode, Louise and Elise generated a brief scientific investigation which contained 
only one category. Inquiry, directly relating to the process of scientific investigation. Other 
categories included Technical Problems which were outside the control of the students and 
included hardware and software problems such as the computer freezing and limited 
memory which resulted in slowness of the software. A third category. Improving Methods, 
consisted of interactions during which Louise and Elise learned how to use and navigate the 
program more effectively. A fourth category. Writing the Investigation, consisted of the 
students completing a report of their investigation from the supplied curriculum support 
material. For example. Instance #2 for Improving Methods (see "Use of" highlighted in Figure 
3, revealed that Louise and Elise were able refine the presentation of their data by using a 
graph instead of a table. This suggests the development of higher-order thinking as result of 
interacting with the inquiry-based interactive multimedia program. The description for the 
"Use of" box in Figure 3 are displayed when the box is double clicked. The description for 
this box is shown in Figure 3. 



Investigation I 

Improving Methods: Instance #2 

(Index: 3:31 - 4:06 minutes; duration: 35 seconds) 

Use of graph to improve data display 

Louise is dominant. Louise came up with a new idea how to use the graph instead of 
presenting the data in a table. 

Louise: You don't even to need to collect the information at all. You put the species Adelie 
Penguin and it came with 37, in and it come up with the number and then you can do the 
graph straight away of bird behaviour... 

Elise: ... and now instead of looking at all the voyages we will do the right course. 



Figure 3: The description in the text window from Improving Methods: Instance #2 for 

Investigation I by Louise and Elise 

The complexity of profiles for investigations increased over time. More efficient use of the 
multimedia program and higher order thinking were reflected in more complex designs of 
investigations with each subsequent use of the program. These changes are evident when 
Investigations I and II are compared (see Figures 2 and 4). 
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Figure 4: Complete VideoSearch display summary of collated data for Investigation II 

by Lousie and Elise 

Students Reflection of Higher Order Thinking Skills 

To probe students' higher order thinking, an interview was conducted with Louise and 
Elise. The questions put to the students were based on theoretical understandings of the 
constructs of higher order thinking (see Resnick & Resnick, 1992; Newmann, 1990). 
Observed categories and instances of those observed categories show the students' 
knowledge and reflections on their use and development of higher order thinking skills (see 
Figure 5). 
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Figure 5: Observed categories for the students' interview 

While the students did not explicitly articulate their development of higher order thinking, 
the way they expressed their experiences showed they were using and had developed those 
skills over the month. The students demonstrated a developing sense of inference, 
interpretation, analysis, judgement, active mental construction and synthesis, all of which 
are essential elements of higher order thinking (Resnick & Resnick, 1992; Newmann, 1990). 
When asked during the interview to reflect on their own learning, the students chose to 
focus on the process of the scientific investigation rather than the content of the program: 

Dorit: What did you learn from the program? 

Louise: How to conduct an investigation. 

Elise: How to go about collecting information. BEST COPY AVAILABLE 
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Louise: Yeah, what information was needed specifically for a question 

Elise: Breaking down the question to find out what you needed in order to answer it 
and going and finding out and then how you followed it up. . . access information. 

(Reflection about your own learning: Instance # 1 ; Index: 1 min 2 3.69 seconds) 

In the above instance, the students demonstrated skills in interpreting and analysing the 
question and making judgements about the relevance of available information. Further into 
this instance, when asked by the researcher whether they usually did these things in their 
science classroom, Louise and Elise highlighted how the process of scientific investigation 
enabled them to focus on the content of the program to answer the question: 

Dorit: Do you usually do these things in science? 

Elise: Y-e-s, we did those but not so specific, (looking at Louise) 

Louise: No it's different. Here you have the questions and you have to break it 
down, to see what information you need. But if you look at the question you might 
think I need environment and sea conditions but when you look at the map and do it 
you need more information than that. So you can see from looking at the map you 
need iceberg and everything else to do with it. 

Elise jumping in. Both girls look enthusiastic and volunteered information. 

Elise: In the classroom they give you an aim and methods... but here you have to 
find out your own way to answer the questions. 

Dorit: Did you have a feeling of scientific investigation here? 

Elise & Louise: Yes, we did. 

(Reflection about your own learning: Instance #2; Index: 1 min 50.69 seconds) 

Students' found working together supported their learning by providing multiple 
perspectives on the investigation: 

Elise: . . . you can talk and get different points of view. 

(Working together: Instance #; Index: 5 mins 14.58 seconds) 

Students also engaged in active mental construction of the designing and conducting 
scientific investigations based on the multimedia program: 

Louise: You've got to do it all by yourself. You haven't got the question, hypothesis 
and so on. So you have to think about your questions. You think of variables, and 
data to collect. 

Dorit: There is a lot of your input and involvement in the program? 

Elise: You are actually happy when you're doing it. When you're doing it then you 
understand it better. 

Dorit: Then, was it your own input more than what the program gave you? 

Louise & Elise: Yes. The program helped to think 

(Instance #2; Index: 6 mins 20.39 seconds) 
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Inferential thinking was also evident in Louise and Elise's reflections and actions during the 
interview: 

Dorit: Yourself as a learner. . . What's going on inside your mind? 

Elise: . . . you have to think about each step before you doing something, think about 
where it is going to get you and is it going to answer my question? Is there a better 
way to do it? Is there a quicker way? Maybe more effective? So you think about it 
this way. 

Louise jumping in impatiently. 

Louise: When you get a question you go straight into something, and don't think 
about other options. If there's an easy way of doing it, we wouldn't think about it. 

Elise: Both times we collected a lot of data and found later that we could just go into 
the graph. 

(Yourself as a learner: Instance # 1 ; Index: 2 mins 43.35 seconds) 

After having experienced a trial and error approach to scientific investigation and, according 
Louise, not having considered other options, Elise emphasised they had learned to judge 
and to think about each step in the process before they began it. They began to look for 
"better", "quicker" and "more effective" ways to conduct investigations, something they 
had not previously attempted. This demonstrates a, wide range of higher order thinking 
skills which was directly enabled by the use of the inquiry-based interactive multimedia 
program while working co-operatively with another student. Students reflected on the 
nature of the program, the process of their own learning, and their perceptions of 
themselves learners. 

Analysing Social Negotiation 

Social negotiation between pairs of students provided ongoing and immediate feedback on 
the students' higher order thinking skills. Open-ended challenges were provided during 
interactions with the program and curriculum material which included a guide to the 
program and content-based questions. A presentation by pairs of students to the whole class 
informed the teachers and other students about the extent to which participants were 
successful in using their higher order thinking skills. 

The complexity of the visual profile for Investigation II in Figure 4 readily contrasts with the 
relatively simple visual profile for Investigation I in Figure 2. The more complex visual 
profile for Investigation II was created by an increased number of observed categories, up 
nine to 13, from four in Investigation I. On-task interactions involving social negotiation 
between the students and the multimedia program increased over the month from 7:48.79 
minutes during Investigation I to 26:25.00 minutes for Investigation II. Similarly, the content 
of interactions in each instance of observed categories was more elaborate and the duration 
of the instances increased during the same time (see Figure 6). 
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Investigation II 

New Investigation: Instance #2 

(Index: 12:03 - 15:40; duration: 3 minutes 37 seconds) 

Both students jointly conducted the investigation. 

Louise suggested the idea followed by Elise who explained the concept on which they were 
working. 

Louise and Elise swaped between screens. 

Dorit (researcher): What are you doing now? 

Louise: Making a histogram of sea state and... 

(some negotiation between the girls about the selection of variables) 

Elise: The ice berg will show on the graph. 

Louise agreed: We'll choose a variable of sea state against the frequency to see if the sea 
state affects the number of ice bergs. 

The program generated a graph but the students didn't stop to discuss it. 

The TOTAL PHYSICAL OBS. axis on the graph confused Elise. 

Louise: We need to make two different graphs. 

Dorit: You can select scatterplot. 

Meanwhile, they selected another variable, sea ice and got a different graph, a histogram 
with three bars. They found it difficult to interpret the graph and Louise continued to 
design a scatterplot. Louise then selected Ice berg against Sea State 

Louise: Now we can see if there is a strong relationship... so all the dots will be together. 



Figure 6: The full description of New Investigation: Instance #2, one instance within 

Investigation II by Louise and Elise 

After having interacted with the program and socially negotiated for one month, student 
pairs presented their experiences of conducting the investigations to the whole class. This 
exercise provided further data about students' use of higher order thinking and socially 
negotiation skills as they were required to reflect on and share their experiences. 

Analysing Students' Presentation for the Development of Higher-Order Thinking Skill 

After one month of using the program, student pairs presented one example of an 
investigation to the whole class. At the introduction of this presentation, Elise chose to 
reflect on her work and chose to compare it with her experiences of conducting science 
investigations in a more didactic learning environment with which she was more familiar. 
Viewing her presentation using VideoSearch allowed the researcher to chronologically index 
and extract specific information and reflect on students development of higher level 
thinking skills. For example, for the observed category. Presentation, the following instance. 
Instance #1 commences at 3.82 seconds into one episode on the video. In this instance, Elise 
expressed her reflections by comparing her experiences in the inquiry-based interactive 
multimedia learning environment to those in the real world: 



Elise: In the real world. . . let's say we have to do something in Biology, we will have 
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lots of information, different ways of collecting it, different ways of graphing it and 
collating it, different ways of showing it and collating it. 

( Presentation : Instance#l ; Index: 3.82 seconds). 

VideoSearch further enabled the researcher to chronologically locate a later comparison 
made by Elise. In this comparison, Elise considers how her experiences over the previous 
month compared with her usual experiences in the science classroom: 

Elise: That sort of taught us how to do that... taught us how we needed to make 
decisions in order to get the best result we could. As in the classroom we were given 
the straight thing, it's like the exercise that already set for us, we have the info we are 
told how to use it. All you have to do is do that, what's written down. 

( Presentation : Instance #2; Index: 17.19 seconds). 

An even later instance in which Elise compared her previous month's experiences as a 
learner in a novel learning environment was also able to be chronologically located and 
connected with the two earlier comparisons to provide a more contextualised overview of 
Elise's reflections: 

Elise: It's sort of developing different skills. . . made us think more about it, and you 
are more interested in it all because you made more your own decisions 

(. Presentation : Instance #3; Index: 34.15 seconds). 

Such reflections have been shown to be essential in the development of higher order 
thinking skills (Ennis, 1987). The fact that the previous month's experiences in the science 
classroom working with the program resulted in quite deep and spontaneous reflections 
about her own learning augurs well for the development of her higher order thinking skills. 
Being able to chronologically index and retrieve each of these instances enables the 
researcher to provide highly contextualised, in-depth insight into her thinking which is not 
available when using other qualitative data analysis tools currently available. Further 
insights were gained through an interview with the students after they had been working 
with the program for one month. 

Discussion 

When students presented their investigations to the whole class, the role of higher order 
thinking skills became evident. Through reflecting on their experiences, students were 
better able to make inferences about the information provided, to interpret new information 
about the program and its content, to analyse available information, make judgements about 
its relevance and synthesise what they had learned to generate new knowledge about the 
Antarctic environment. The development of higher order thinking was even more evident 
during an interview with the two students who were able to articulate their own thinking 
strategies. After probing by the researcher, they were able to think about themselves as 
learners and clearly able to reflect on their own learning which they found intrinsically 
motivating. Working co-operatively was fundamental to the whole process and appears to 
have enhanced the students' abilities to use higher order thinking skills and to reflect on that 
use. 

Students were able to generate questions as part of a scientific investigation through social 
negotiation with their partners. It seems that the reason for the difference stemmed from the 
planning for social negotiation as an integral part of the learning environment. The 
researcher interacting with the students was as important as the students working together, 
negotiating and co-operating while interacting with the program. This is evident in the type 
and number of categories present during the second investigation but absent in the first 
investigation. In previous studies in this area by Maor (1993) and Maor and Taylor (1995) 
social negotiation occurred on an ad hoc basis. It was noticed that during infrequent social 
negotiations students asked more creative questions during investigations. This observation 
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led to social negotiation being actively planned for in the current study. Compared to the 
earlier research, students in this study were able to generate more relevant and complex 
questions. 

The initial scientific investigation conducted by the students in this study, was based on 
their limited inquiry skills (e.g., answering a question from curriculum materials, collecting 
data, improving methods of data collection). However, subsequent investigations revealed 
more complexity and more variety in the students' range of inquiry skills (e.g. generating a 
question, generating and verifying an hypothesis and designing and investigation). This 
suggests students progressed from low levels of inquiry skills to higher level skills during 
the study. 

The multimedia program, by providing multiple representations of reality and the natural 
complexity of the real world, enabled students, working collaboratively, to focus on 
knowledge construction while conducting authentic scientific investigations. Video taping 
students interactions while conducting scientific investigations using the multimedia 
program provided a means for better understanding the context of the social processes of 
students' learning. 

The researcher's previous experience in conducting interpretive research involved analysing 
data essentially removed from the full context in which it occurred despite thick description 
and using triangulation (Guba & Lincoln, 1989; Erickson, 1986). With the time required to 
analyse qualitative data, there is a risk of losing or distorting both the data and its 
interpretation thus reducing the trustworthiness of the research (Guba &Lincoln, 1989). This 
situation is created by a reduced number of dimensions which is necessary to maintain 
control over the data. These fewer dimensions can then be coded for subsequent analyses. 
However, with digital video analysis it is possible to code and analyse multiple dimensions 
in the data (e.g., dialogue, researcher comments, etc). Digital video analysis allowed the 
researcher to more finely tune the data analysis and therefore extend the researcher's claims 
while ensuring greater accountability. 

However, the time required and the level of intensive analysis may make digital video 
analysis a difficult and expensive process to undertake in its current form. For a researcher 
to be able to use the program effectively they would need to devote the time required, to 
learn digital video coding techniques, have access to sufficient disk storage space and be 
able to use a digital video camera. If it is possible to overcome these limitations, being able 
to zoom in on and analyse very small episodes of students' learning means researchers can 
access and analyse additional dimensions of students' learning. Future researchers could 
also focus on dimensions not addressed in this research, for example, the role of non-verbal 
communication in student learning in the classroom. This study has provided an initial 
profile of a scientific investigation and provides a means for researchers to develop a corpus 
of profiles of investigations. This may help teachers and teacher educators to bridge the gap 
between theory and practice in the science classroom. 
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